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Abstract 

Biological systems produce outputs in response to variable inputs. Input-output relations tend to follow a few regular 
patterns. For example, many chemical processes follow the S-shaped Hill equation relation between input 
concentrations and output concentrations. That Hill equation pattern contradicts the fundamental Michaelis-Menten 
theory of enzyme kinetics. I use the discrepancy between the expected Michaelis-Menten process of enzyme kinetics 
and the widely observed Hill equation pattern of biological systems to explore the general properties of biological 
input-output relations. I start with the various processes that could explain the discrepancy between basic chemistry 
and biological pattern. I then expand the analysis to consider broader aspects that shape biological input-output 
relations. Key aspects include the input-output processing by component subsystems and how those components 
combine to determine the system's overall input-output relations. That aggregate structure often imposes strong 
regularity on underlying disorder. Aggregation imposes order by dissipating information as it flows through the 
components of a system. The dissipation of information may be evaluated by the analysis of measurement and 
precision, explaining why certain common scaling patterns arise so frequently in input-output relations. I discuss how 
aggregation, measurement and scale provide a framework for understanding the relations between pattern and 
process. The regularity imposed by those broader structural aspects sets the contours of variation in biology. Thus, 
biological design will also tend to follow those contours. Natural selection may act primarily to modulate system 
properties within those broad constraints. 

Reviewers: This article was reviewed by Eugene Koonin, Georg Luebeck and Sergei Maslov. 
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comments section. 

Introduction 

Cellular receptors and sensory systems measure input sig- 
nals. Responses flow through a series of downstream pro- 
cesses. Final output expresses physiological or behavioral 
phenotype in response to the initial inputs. A system's 
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overall input-output pattern summarizes its biological 
characteristics. 

Each processing step in a cascade may ultimately be 
composed of individual chemical reactions. Each reac- 
tion is itself an input-output subsystem. The input signal 
arises from the extrinsic spatial and temporal fluctua- 
tions of chemical concentrations. The output follows from 
the chemical transformations of the reaction that alter 
concentrations. The overall input-output pattern of the 
system develops from the signal processing of the com- 
ponent subsystems and the aggregate architecture of the 
components that form the broader system. 

Many fundamental questions in biology come down 
to understanding these input-output relations. Some sys- 
tems are broadly sensitive, changing outputs moderately 
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over a wide range of inputs. Other systems are ultra- 
sensitive or bistable, changing very rapidly from low to 
high output across a narrow range of inputs [1]. The 
Hill equation describes these commonly observed input- 
output patterns, capturing the essence of how changing 
inputs alter system response [2]. 

I start with two key questions. How does the commonly 
observed ultrasensitive response emerge, given that clas- 
sical chemical kinetics does not naturally lead to that 
pattern? Why does the very simple Hill equation match 
so well to the range of observed input-output relations? 
To answer those questions, I emphasize the general pro- 
cesses that shape input-output relations. Three aspects 
seem particularly important: aggregation, measurement, 
and scale. 

Aggregation combines lower-level processes to produce 
the overall input-output pattern of a system. Aggregation 
often transforms numerous distinct and sometimes disor- 
dered lower-level fluctuations into highly regular overall 
pattern [3]. One must understand those regularities in 
order to analyze the relations between pattern and pro- 
cess. Aggregate regularity also imposes constraints on 
how natural selection shapes biological design [4]. 

Measurement describes the information captured from 
inputs and transmitted through outputs. How sensitive 
are outputs to a change in inputs? The overall pattern of 
sensitivity affects the information lost during measure- 
ment and the information that remains invariant between 
input and output. Patterns of sensitivity that may seem 
puzzling or may appear to be specific to particular mech- 
anisms often become simple to understand when one 
learns to read the invariant aspects of information and 
measurement. Measurement also provides a basis for 
understanding scale [5]. 

Scale influences the relations between input and output 
[6]. Large input typically saturates a system, causing out- 
put to become insensitive to further increases in input. 
Saturated decline in sensitivity often leads to logarithmic 
scaling. Small input often saturates in the other direction, 
such that output changes slowly and often logarithmically 
in response to further declines in input. The Hill equation 
description of input-output patterns is simply an expres- 
sion of logarithmic saturation at high and low inputs, with 
an increased linear sensitivity at intermediate input levels. 

High input saturates output because maximum output is 
intrinsically limited. By contrast, the commonly observed 
logarithmic saturation at low input intensity remains a 
puzzle. The difficulty arises because typical theoretical 
understanding of chemical kinetics predicts a strong and 
nearly linear output sensitivity at low input concentra- 
tions of a signal [7]. That theoretical linear sensitivity 
of chemical kinetics at low input contradicts the widely 
observed pattern of weak logarithmic sensitivity at low 
input. 



I describe the puzzle of chemical kinetics in the next 
section to set the basis for a broader analysis of input- 
output relations. I then connect the input-output relations 
of chemical kinetics to universal aspects of aggregation, 
measurement, and scale. Those universal properties of 
input-output systems combine with specific biological 
mechanisms to determine how biological systems respond 
to inputs. Along the way, I consider possible resolutions 
to the puzzle of chemical kinetics and to a variety of other 
widely observed but unexplained regularities in input- 
output patterns. Finally, I discuss the ways in which regu- 
larities of input-output relations shape many key aspects 
of biological design. 

Review 

The puzzle of chemical kinetics 

Classical Michaelis-Menten kinetics for chemical reac- 
tions lead to a saturating relationship between an input 
signal and an output response [7]. The particular puz- 
zle arises at very low input, for which Michaelis-Menten 
kinetics predict a nearly linear output response to tiny 
changes in input. That sensitivity at low input means that 
chemical reactions would have nearly infinite measure- 
ment precision with respect to tiny fluctuations of input 
concentration. Idealized chemical reactions do have that 
infinite precision, and observations may follow that pat- 
tern if nearly ideal conditions are established in laboratory 
studies. By contrast, the actual input-output relations of 
chemical reactions and more complex biological signals 
often depart from Michaelis-Menten kinetics. 

Many studies have analyzed the contrast between 
Michaelis-Menten kinetics and the observed input-output 
relations of chemical reactions [2]. I will discuss some of 
the prior studies in a later section. However, before con- 
sidering those prior studies, it is useful to have a clearer 
sense of the initial puzzle and of alternative ways in which 
to frame the problem. 

Example of Michaelis-Menten kinetics 

I illustrate Michaelis-Menten input-output relations with 
a particular example, in which higher input concentra- 
tion of a signal increases the transformation of an inac- 
tive molecule to an active state. Various formulations of 
Michaelis-Menten kinetics emphasize different aspects of 
reactions [7]. But those different formulations all have the 
same essential mass action property that assumes spa- 
tially independent concentrations of reactants. Spatially 
independent concentrations can be multiplied to calcu- 
late the spatial proximity between reactants at any point in 
time. 

In my example, a signal, S, changes an inactive reactant, 
R, to an active output, A, in the reaction 

S + R^S + A, 
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where the rate of reaction, g, can be thought of as the 
signal gain. In this reaction alone, if S > 0, all of the 
reactant, R, will eventually be transformed into the active 
form, A. (I use roman typeface for the distinct reactant 
species and italic typeface for concentrations of those 
reactants.) However, I am particularly interested in the 
relation between the input signal concentration, S, and 
the output signal concentration, A. Thus, I also include 
a back reaction, in which the active form, A, sponta- 
neously transforms back to the inactive form, R, expressed 
as 




The reaction kinetics follow 

A = gS(N -A)- SA, (1) 

in which the overdot denotes the derivative with respect to 
time, and N = R + A is the total concentration of inactive 
plus active reactant molecules. We find the equilibrium 
concentration of the output signal, A*, as a function of the 
input signal, S, by solving A = 0, which yields 

A* = N ( — ^— ) , (2) 
\m+SJ 

in which m = S/g is the rate of the back reaction rela- 
tive to the forward reaction. Note that S/(m + S) is the 
equilibrium fraction of the initially inactive reactant that 
is transformed into the active state. At S = m, the input 
signal transforms one-half of the inactive reactant into the 
active state. 

Figure 1 shows the consequence of this type of 
Michaelis-Menten kinetics for the relation between the 
input signal and the output signal. At low input signal 
intensity, S ^ 0, the output is strongly (linearly) sensitive 
to changes in input, with the output changing in propor- 
tion to S. At high signal intensity, the output is weakly (log- 
arithmically) sensitive to changes in input, with the output 
changing in proportion to leges'). The output saturates at 
A* — > N as the input increases. 

The Hill equation and observed input-output patterns 

Observed input-output patterns often differ from the sim- 
ple Michaelis-Menten pattern in Figure 1. In particular, 
output is often only weakly sensitive to changes in the 
input signal at low input intensity. Weak sensitivity at low 
input values often means that output changes in propor- 
tion to log(S) for small S values, rather than the linear rela- 
tion between input and output at small S values described 
by Michaelis-Menten kinetics. 

The Hill equation preserves the overall Michaelis- 
Menten pattern but alters the sensitivity at low inputs to 
be logarithmic rather than linear. Remarkably, the pattern 
of curve shapes for most biochemical reactions and more 
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Figure 1 Michaelis-Menten signal transmission. The reaction 




dynamics transform the concentration of the input signal, S, into the 


equilibrium output signal, A*, as given by Eq. (2). Half maximal output 


occurs at input S = m. The total reactant available to be transformed 


is N. 







general biological input-output relations fit reasonably 
well to the Hill equation 

\ m K + x K I 

or to minor variants of this equation (Table 1). The input 
intensity is x, the measured output is y, half of maximal 
response is x = m, the shape of the response is deter- 
mined by the Hill coefficient k, and the response saturates 
asymptotically at b for increasing levels of input. 

We can simplify the expression by using the substitu- 
tions y = y/b, in which y is the fraction of the maximal 
response, and X = x/m, in which x is the ratio of the input 
to the value that gives half of the maximal response. The 
resulting equivalent expression is 

x k 

(4) 

Figure 2 shows the input-output relations for different 
values of the Hill coefficient, k. For k = 1, the curve 
matches the Michaelis-Menten pattern in Figure 1. An 
increase in k narrows the input range over which the out- 
put responds rapidly (sensitively). For larger values of k, 
the rapid switch from low to high output response is often 
called a bistable response, because the output state of 
the system switches in a nearly binary way between low 
output, or "OFF", and high output, or "ON" A bistable 
switching response is effectively a biological transistor 
that forms a component of a biological circuit [13]. Insta- 
bility is sometimes called ultrasensitivity, because of the 
high sensitivity of the response to small changes in inputs 
when measured over the responsive range [14]. 

At the k = 1 case of Michaelis-Menten, the output 
response is linearly sensitive to very small changes at very 
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Table 1 Conceptual foundations 

Psychophysics studies human perception of quantity, such as loudness, temperature or pressure. The early work of Weber and Fechner 
suggested that perception scales logarithmically: for a given stimulus (input), the perception of quantity (output) changes logarithmically. 
That work led to modern analysis of measurement and scale. 

This article analyzes biological input-output relations. My examples focus on biochemistry. I chose that focus for two reasons. First, 
most biological input-output relations may ultimately be reducible to cascades of biochemical component reactions. The problem then 
becomes how to relate the biochemical components and their connections to overall system function. That relation between biochem- 
istry and system function is the core of modern systems biology. Second, the sharp distinction between classical Michaelis-Menten 
chemical kinetics and the observed patterns of logarithmic scaling in both biochemistry and perception provides a good entry into the 
unsolved puzzles of the subject and the potential value of my perspective. 

Although I focus on biochemistry, my approach derives from other topics. I borrow the deep conceptual foundations of measurement 
from psychophysics, the principles of aggregation from statistical mechanics, and aspects of information theory that originally developed 
in studies of communication. My view is that biological input-output relations can only be understood in terms of aggregation, measure- 
ment and information. In this article, I evoke those principles indirectly by building a series of specific analyses of biochemistry and simple 
aspects of systems biology. 

The literatures and conceptual spans are vast for psychophysics, measurement theory, statistical mechanics and information theory. Here, 
I mention a few key entries into each subject. To read this article, it is not necessary to understand all of those topics. But it is necessary to 
seethe project for what it is, an attempt to borrow deep principles from other subjects and apply those principles to biochemical aspects 
of systems biology, to the nature of biological input-output relations, and to the consequences for natural selection and evolutionary 
design. 

Gescheider [8] summarizes aspects of psychophysics related to my discussion of input-output patterns. History and further references 
can be obtained from that work. Certain aspects of measurement theory followed from psychophysics [5,6], The theory developed into a 
broader analysis of the principles of quantity [9-11]. Other branches of measurement theory focus on aspects of precision and calibration 
[12]. 

Statistical mechanics analyzes the ways in which aggregation leads to highly ordered systems arising from disordered underlying compo- 
nents. My usage follows from the proposed unity between information theory and aggregate pattern, which transcends the specifics of 
physical models and instead emphasizes the patterns expressed by probability distributions [3,40]. That Jaynesian perspective describes 
how aggregation dissipates information to expose underlying regularity. Later work [41,42] provided a unified framework for all common 
probability patterns by combining measurement theory with Jaynes' information theory interpretation of statistical mechanics. 



low input signals. Such extreme sensitivity means essen- 
tially infinite measurement precision at tiny input levels, 
which seems unlikely for realistic biological systems. As 
k increases, sensitivity at low input becomes more like a 
threshold response, such that a minimal input is needed 
to stimulate significant change in output. Increasing k 
causes sensitivity to become logarithmic at low input. 
That low input sensitivity pattern can be seen more clearly 
by plotting the input level on a logarithmic scale, as in 
Figure 3. 




Input signal, x 

Figure 2 Hill equation signal transmission. The input signal, x, 
leads to the output, y, as given by Eq. (4). The curves of increasing 
slope correspond to k = 1 , 2, 4, 8. 



Alternative mechanisms for simple chemical reactions 

My goal is to understand the general properties of input- 
output relations in biological systems. To develop that 
general understanding, it is useful to continue with study 
of the fundamental input-output relations of simple chem- 
ical reactions. Presumably, most input-output relations of 




Input signal, log 10 x 

Figure 3 An increasing Hill coefficient, k, causes logarithmic 
sensitivity to low input signals. At k = 1 (left curve), the sensitivity 
is linear with a steady increase in output even at very low input levels, 
implying infinite precision. As k increases, sensitivity at low input 
declines, and the required threshold input level becomes higher and 
sharper to induce an output response of 1% of the maximum 
(y = 0.01). The curves of increasing slope correspond to k = 1,2,4,8 
in Eq. (4), with logarithmic scaling of the inputx plotted here. 
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systems can ultimately be decomposed into simple com- 
ponent chemical reactions. Later, I will consider how 
the combination of such components influences overall 
system response. 

Numerous studies of chemical kinetics report Hill coef- 
ficients k > 1 rather than the expected Michaelis-Menten 
pattern k = 1. Resolution of that puzzling discrepancy 
is the first step toward deeper understanding of input- 
output patterns (Table 2). Zhang et al. [2] review six 
specific mechanisms that may cause k > 1. In this section, 
I briefly summarize several of those mechanisms. See 
Zhang et al. [2] for references. 

Direct multiplication of signal input concentration 

Transforming a single molecule to an active state may 
require simultaneous binding by multiple input signal 
molecules. If two signal molecules, S, must bind to a sin- 
gle inactive reactant, R, to form a three molecule complex 
before transformation of R to the active state, A, then we 
can express the reaction as 

S + S + R^SSR^ S + S+A, 

which by mass action kinetics leads to the rate of change 
in A as 

A =gS 2 (N-A) - SA, 



in which N = R + A is the total concentration of the inac- 
tive plus active reactant molecules, and the back reaction 
A -> R occurs at rate S. The equilibrium input-output 
relation is 

a*=n(^-,), 

\m 2 +S 2 J 

which is a Hill equation with k = 2. The reaction 
stoichiometry, with two signal molecules combining in 
the reaction, causes the reaction rate to depend multi- 
plicatively on signal input concentration. Other simple 
schemes also lead to a multiplicative effect of signal 
molecule concentration on the rate of reaction. For exam- 
ple, the signal may increase the rates of two sequential 
steps in a pathway, causing a multiplication of the sig- 
nal concentration in the overall rate through the multiple 
steps. Certain types of positive feedback can also amplify 
the input signal multiplicatively. 

Saturation and loss of information in multistep reaction 
cascades 

The previous section discussed mechanisms that multi- 
ply the signal input concentration to increase the Hill 
coefficient. Multiplicative interactions lead to logarithmic 
scaling. The Hill equation with k > 1 expresses loga- 
rithmic scaling of output at high and low input levels. 
I will return to this general issue of logarithmic scaling 
later. The point here is that multiplication is one sufficient 



Table 2 Literature related to the Hill equation 

The Hill equation or related expressions form a key part of analysis in many areas of chemistry, systems biology and pharmacology. Each 
subdiscipline has its own set of isolated conceptual perspectives and mutual citation islands. Here, I list a few publications scattered over 
that landscape. My haphazard sample provides a sense of the disconnected nature of the topic. I conclude from this sample that the 
general form of input-output relations is widely recognized as an important problem and that no unified conceptual approach exists. 

Cornish-Bowden's [7] text on enzyme kinetics frequently uses the Hill coefficient to summarize the relation between input concentrations 
and the rate of transformation to outputs. That text applies both of the two main approaches. First, the Hill equation simply provides 
a description of how changes in input affect output independently of the underlying mechanism. Second, numerous specific models 
attempt to relate particular mechanisms to observed Hill coefficients. Zhang et al. [2] provide an excellent, concise summary of specific 
biochemical mechanisms, including some suggested connections to complex cellular phenotypes. 

Examples of the systems biology perspective include Kim & Ferrell [15], Ferrell [16], Cohen-Saidon et al. [17], Goentoro& Kirschner [18] and 
Goentoro et al. [19]. Alon's [20] leading text in systems biology discusses the importance of the Hill equation pattern, but only considers 
the explicit classical chemical mechanism of multiple binding. Those studies share the view that specific input-output patterns require 
specific underlying mechanisms as explanations. In pharmacology, the Hill equation provides the main approach for describing dose- 
response patterns. Often, the Hill equation is used as a model to fit the data independently of mechanism. That descriptive approach 
probably follows from the fact that many complex and unknown factors influence the relation between dose and response. Alternatively, 
some analyses focus on the key aspect of receptor-ligand binding in the response to particular drugs. Reviews from this area include 
DeLean etal. [21], Weiss [22], Rang [23] and Bindslev [24]. Related approaches arise in the analysis of basic physiology [25]. 

Other approaches consider input-output responses in relation to aggregation of underlying heterogeneity, statistical mechanics or 
aspects of information. Examples include Hoffman & Goldberg [26], Getz& Lansky [27], Kolch et al. [28],Tkacik& Walczak [29] and Marzen 
et al. [30], 

Departures from the mass-action assumption of Michaelis-Menten kinetics can explain the emergence of Hill equation input-output rela- 
tions [31,32]. Many studies analyze the kinetics of diffusion-limited departures from mass action without making an explicit connection to 
the Hill equation [33-35], Modeling approaches in other disciplines also consider the same problem of departures from spatial uniformity 
[36-38] 

Studies often use the Hill equation or similar assumptions to describe the shapes of input-output functions when building models of 
biochemical circuits [13,39], Those studies do not make any mechanistic assumptions about the underlying cause of the Hill equation 
pattern. Rather, in order to build a model circuit for regulatory control, one needs to make some assumption about input-output relations. 
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way to achieve logarithmic scaling. But multiplication is 
not necessary. Other nonmultiplicative mechanisms that 
lead to logarithmic scaling can also match closely to the 
Hill equation pattern. This section discusses two examples 
covered by Zhang et al. [2], 

Repressor of weak input signals The key puzzle of the 
Hill equation concerns how to generate the logarithmic 
scaling pattern at low input intensity. The simplest non- 
multiplicative mechanism arises from an initial reaction 
that inactivates the input signal molecule. That prepro- 
cessing of the signal intensity can create a filter that 
logarithmically reduces signals of low intensity. Suppose, 
for example, that the repressor may become saturated 
at higher input concentrations. Then the initial reaction 
filters out weak, low concentration, inputs but passes 
through higher input concentrations. 

Consider a repressor, X, that can bind to the signal, S, 
transforming the bound complex into an inactive state, I, 
in the reaction 

S + X^I. 

P 

One can think of this reaction as a preprocessing filter 
for the input signal. The kinetics of this input preproces- 
sor can be expressed by focusing on the change in the 
concentration of the bound, inactive complex 

i=y(S-I)(X-I)-pi. (5) 

The signal passed through this preprocessor is the 
amount of S that is not bound in I complexes, which is 
S' = S — I. We can equivalently write / = S' — S. The 
equilibrium relation between the input, S, and the out- 
put signal, S', passed through the preprocessor can be 
obtained by solving 1 = 0, which yields 

S'(X-S + S')-a(S-S') = 0, 

in which a = f}/y. Figure 4a shows the relation between 
the input signal, S, and the preprocessed output, S'. Bound 
inactive complexes, /, hold the signal molecule tightly and 
titrate it out of activity when the breaking up of complexes 
at rate fi is slower than the formation of new complexes at 
rate y, and thus a is small. 

The preprocessed signal may be fed into a standard 
Michaelis-Menten type of reaction, such as the reaction in 
Eq. (1), with the preprocessed signal S' driving the kinet- 
ics rather than the initial input, S. The reaction chain from 
initial input through final output starts with an input con- 
centration, S, of which S' passes through the repressor 
filter, and S' stimulates production of the active output sig- 
nal concentration, A*. Figure 4b shows that titration of the 
initial signal concentration, S, to a lower pass-through sig- 
nal concentration, S', leads to low sensitivity of the final 



output, A*, to the initial signal input, S, as long as the sig- 
nal concentration is below the amount of the repressor 
available for titration, X. 

When this signal preprocessing mechanism occurs, the 
low, essentially logarithmic, sensitivity to weak input 
signals solves the puzzle of relating classical Michaelis- 
Menten chemical kinetics to the Hill equation pattern for 
input-output relations with k > 1. The curves in Figure 4b 
do not exactly match the Hill equation. However, this 
signal preprocessing mechanism aggregated with other 
simple mechanisms can lead to a closer fit to the Hill 
equation pattern. I discuss the aggregation of different 
mechanisms below. 

This preprocessed signal system is associated with clas- 
sical chemical kinetic mechanisms, because it is the deter- 
ministic outcome of a simple and explicit mass action 
reaction chain. However, the reactions are not inher- 
ently multiplicative with regard to signal input intensity. 
Instead, preprocessing leads to an essentially logarithmic 
transformation of scaling and information at low input 
signal intensity. 

This example shows that the original notion of mul- 
tiplicative interactions is not a necessary condition for 
Hill equation scaling of input-output relations. Instead, 
the Hill equation pattern is simply a particular expres- 
sion of logarithmic scaling of the input-output relation. 
Any combination of processes that leads to similar log- 
arithmic scaling provides similar input-output relations. 
Thus, the Hill equation pattern does not imply any partic- 
ular underlying chemical mechanism. Rather, such input- 
output relations are the natural consequence of the ways 
in which information degrades and is transformed in rela- 
tion to scale when passed through reaction sequences that 
act as filters of the input signal. 

Opposing forward and back reactions The previous 
section showed how a repressor can reduce sensitivity to 
low intensity input signals. A similar mechanism occurs 
when there is a back reaction. For example, a signal may 
transform an inactive reactant into an active form, and 
a back reaction may return the active form to the inac- 
tive state. If the back reaction saturates at low signal input 
intensity, then a rise in the signal from a very low level will 
initially cause relatively little increase in the concentration 
of the active output, inducing weak, logarithmic sensitiv- 
ity to low input signal intensity. In effect, the low input is 
repressed, or titrated, by the strong back reaction. 

This opposition between forward and back reactions 
was one of the first specific mechanisms of classical chem- 
ical kinetics to produce the Hill equation pattern in the 
absence direct multiplicative interactions that amplify the 
input signal [14]. In this section, I briefly illustrate the 
opposition of forward and back reactions in relation to the 
Hill equation pattern. 
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Input signal, S 

Figure 4 Preprocessing of an input signal by a repressor reduces sensitivity of output to low input intensity signals, (a) Equilibrium 
concentration of processed signal, S', in relation to original signal input intensity, 5, obtained by solution of Eq. (5). The four curves from bottom to 
top show decreasing levels of signal titration by the repressor for the parameter values a = 0.01,0.1,0.5, 1000. The top curve alters the initial signal 
very little, so that 5' a* 5, showing the consequences of an unfiltered input signal. (b)The processed input signal, 5', is used as the input to a 
standard Michaelis-Menten reaction kinetics process in Eq. (1), leading to an equilibrium output, A*. The curves from bottom to top derive from the 
corresponding preprocessed input signal from the upper panel. 



In the forward reaction, a signal, S, transforms an inac- 
tive reactant, R, into an active state, A. The back reaction 
is catalyzed by the molecule B, which transforms A back 
into R. The balancing effects of the forward and back reac- 
tions in relation to saturation depend on a more explicit 
expression of classical Michaelis-Menten kinetics than 
presented above. In particular, let the two reactions be 



S + R^SR-^S + A 

s 

B + A^BA^B + R 

d 

in which these reactions show explicitly the intermedi- 
ate bound complexes, SR and BA. The rate of change in 



the output signal, A, when the dynamics follow classical 
equilibrium Michaelis-Menten reaction kinetics, is 

A= * So {^)-° Bo {^h)' (6) 

in which So includes the concentrations of both free signal, 
S, and bound signal, SR. Similarly, Bo includes the concen- 
trations of both free catalyst, B, and bound catalyst, BA. 
The half-maximal reaction rates are set by m = S/g and 
^ = d/y. The degree of saturation depends on the total 
amount of reactant available, N = R + A, relative to the 
concentrations that give the half-maximal reaction rates, 
m and fi. 

When the input signal, So, is small, the back reaction 
dominates, potentially saturating the forward rate as 7? 
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becomes large. Figure 5 shows that the level of satura- 
tion sets the input-output pattern, with greater saturation 
increasing the Hill coefficient, k. 

Alternative perspectives on input-output relations 

In the following sections, I discuss alternative mechanisms 
that generate Hill equation patterns. Before discussing 
those alternative mechanisms, it is helpful to summa- 
rize the broader context of how biochemical and cellular 
input-output relations have been studied. 

Explicit chemical reaction mechanisms 

The prior sections linked simple and explicit chemi- 
cal mechanisms to particular Hill equation patterns of 
inputs and outputs. Each mechanism provided a distinct 
way in which to increase the Hill coefficient above one. 
Many key reviews and textbooks in biochemistry and sys- 
tems biology emphasize that higher Hill coefficients and 
increased input-output sensitivity arise from these simple 
and explicit deterministic mechanisms of chemical reac- 
tions [2,7,20]. The idea is that a specific pattern must 
be generated by one of a few well-defined and explicit 
alternative mechanisms. 

Explicit chemical reaction mechanisms discussed earlier 
include: binding of multiple signal molecules to stimu- 
late each reaction; repressors of weak input signals; and 
opposing forward and back reactions near saturation. 
Each of these mechanisms could, in principle, be isolated 
from a particular system, analyzed directly, and linked 
quantitatively to the specific input-output pattern of a sys- 
tem. Decomposition to elemental chemical kinetics and 
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Figure 5 Balance between forward and back reactions leads to a 
high Hill coefficient when the reactions are saturated. The 

equilibrium output signal, A*, is obtained by solving A = 0 in Eq. (6) as 
a function of the input signal, So. The signal is given as 5 = (PSq/oBq. 
The total amount of reactant is N = R + A. The half-maximal 
concentrations are set to m = /x = 1 . The three curves illustrate the 
solutions for N = 1, 10, 100, with increasing Hill coefficients for higher 
N values and greater reaction saturation levels. 



direct quantitative analysis would link observed pattern to 
an explicit mechanistic process. 

The Hill equation solely as a description of observed pattern 

In the literature, the Hill equation is also used when build- 
ing models of how system outputs may react to various 
inputs (Table 2). The models often study how combi- 
nations of components lead to the overall input-output 
pattern of a system. To analyze such models, one must 
make assumptions about the input-output relations of the 
individual components. Typically, a Hill equation is used 
to describe the components' input-output functions. That 
description does not carry any mechanistic implication. 
One simply needs an input-output function to build the 
model or to describe the component properties. The Hill 
equation is invoked because, for whatever reason, most 
observed input-output functions follow that pattern. 

System-level mechanisms and departures from mass action 

Another line of study focuses on system properties rather 
than the input-output patterns of individual components. 
In those studies, the Hill equation pattern of sensitivity 
does not arise from a particular chemical mechanism in 
a particular reaction. Instead, sensitivity primarily arises 
from the aggregate consequences of the system (Table 2). 
In one example, numerous reactions in a cascade com- 
bine to generate Hill-like sensitivity [39]. The sensitivity 
derives primarily from the haphazard combination of dif- 
ferent scalings in the distinct reactions, rather than a 
particular chemical process. 

Alternatively, some studies assume that chemical kinet- 
ics depart from the classical mass action assumption 
(Table 2). If input signal molecules tend, over the course 
of a reaction, to become spatially isolated from the reac- 
tant molecules on which they act, then such spatial pro- 
cesses often create a Hill-like input-output pattern by 
nonlinearly altering the output sensitivity to changes in 
inputs. I consider such spatial processes as an aggregate 
system property rather than a specific chemical mech- 
anism, because many different spatial mechanisms can 
restrict the aggregate movement of molecules. The aggre- 
gate spatial processes of the overall system determine 
the departures from mass action and the potential Hill- 
like sensitivity consequences, rather than the particular 
physical mechanisms that alter spatial interactions. 

These system-level explanations based on reaction cas- 
cades and spatially induced departures from mass action 
have the potential benefit of applying widely. Yet each 
particular system-level explanation is itself a particular 
mechanism, although at a higher level than the earlier 
biochemical mechanisms. In any actual case, the higher 
system -level mechanism may or may not apply, just as 
each explicit chemical mechanism will sometimes apply to 
a particular case and sometimes not. 
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A broader perspective 

As we accumulate more and more alternative mechanisms 
that fit the basic input-output pattern, we may ask whether 
we are converging on a full explanation or missing some- 
thing deeper. Is there a different way to view the problem 
that would unite the disparate perspectives, without losing 
the real insights provided in each case? 

I think there is a different, more general perspective 
(Table 1). At this point, I have given just enough back- 
ground to sketch that broader perspective. I do so in the 
remainder of this section. However, it is too soon to go 
all the way. After giving a hint here about the final view, I 
return in the following sections to develop further topics, 
after which I return to a full analysis of the broader ways 
in which to think about input-output relations. 

The Hill equation with k > 1 describes weak, logarith- 
mic sensitivity at low input and high input levels, with 
strong and essentially linear sensitivity through an inter- 
mediate range. Why should this log-linear-log pattern be 
so common? The broader perspective on this problem 
arises from the following points. 

First, the common patterns of nature are exactly those 
patterns consistent with the greatest number of alternative 
underlying processes [3,40]. If many different processes 
lead to the same outcome, then that outcome will be com- 
mon and will lack a strong connection to any particular 
mechanism. In any explicit case, there may be a simple and 
clear mechanism. But the next case, with the same pattern, 
is likely to be mechanistically distinct. 

Second, measurement and information transmission 
unite the disparate mechanisms. The Hill equation with 
k > 1 describes a log-linear-log measurement scale 
[41,42]. The questions become: Why do biological sys- 
tems, even at the lowest chemical levels of analysis, often 
follow this measurement scaling? How does chemistry 
translate into the transmission and loss of information in 
relation to scale? Why does a universal pattern of informa- 
tion and measurement scale arise across such a wide range 
of underlying mechanisms? 

Third, this broader perspective alters the ways in 
which one should analyze biological input-output sys- 
tems. In any particular case, specific mechanisms 
remain interesting and important. However, the rela- 
tions between different cases and the overall interpreta- 
tion of pattern must be understood within the broader 
framing. 

With regard to biological design, natural selection works 
on the available patterns of variation. Because certain 
input-output relations tend to arise, natural selection 
works on variations around those natural contours of 
input-output patterns. Those natural contours of pattern 
and variation are set by the universal properties of infor- 
mation transmission and measurement scale. That con- 
straint on variation likely influences the kinds of designs 



created by natural selection. To understand why certain 
designs arise and others do not, we must understand how 
information transmission and measurement scale set the 
underlying patterns of variation. 

I return to these points later. For now, it is useful to 
keep in mind these preliminary suggestions about how the 
various pieces will eventually come together. 

Aggregation 

Most biological input-output relations arise through a 
series of reactions. Initial reactions transform the input 
signal into various intermediate signals, which themselves 
form inputs for further reactions. The final output arises 
only after multiple internal transformations of the initial 
signal. We may think of the overall input-output rela- 
tion as the aggregate consequence of multiple reaction 
components. 

A linear reaction cascade forms a simple type of system. 
Kholodenko el al. [39] emphasized that a cascade tends 
to multiply the sensitivities of each step to determine the 
overall sensitivity of the system. Figure 6 illustrates how 
the input-output relations of individual reactions combine 
to determine the system-level pattern. 

To generate Figure 6, I calculated how a cascade of 12 
reactions processes the initial input into the final output. 
Each reaction follows a Hill equation input-output rela- 
tion given by Eq. (4) with a half-maximal response at m 
and a Hill coefficient of k. The output for each reaction 
is multiplied by a gain, g. The parameters for each reac- 
tion were chosen randomly, as shown in Figure 6a and 
described in the caption. 

Figure 6 shows that a cascade significantly increases the 
Hill coefficient of the overall system above the average 
coefficient of each reaction, and often far above the maxi- 
mum coefficient for any single reaction. Intuitively, the key 
effect at low signal input arises because any reaction that 
has low sensitivity at low input reduces the signal inten- 
sity passed on, and such reductions at low input intensity 
multiply over the cascade, yielding very low sensitivity to 
low signal input. Note in each curve that an input signal 
significantly above zero is needed to raise the output sig- 
nal above zero. That lower tail illustrates the loss of signal 
information at low signal intensity. 

This analysis shows that weak logarithmic sensitiv- 
ity at low signal input, associated with large Hill coef- 
ficients, can arise by aggregation of many reactions. 
Thus, aggregation may be a partial solution to the over- 
all puzzle of log-linear-log sensitivity in input-output 
relations. 

Aggregation by itself leaves open the question of how 
variations in sensitivity arise in the individual reactions. 
Classical Michaelis-Menten reactions have linear sensitiv- 
ity at low signal input, with a Hill coefficient of k = 1. A 
purely Michaelis-Menten cascade with k = 1 at each step 
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Figure 6 Signal processing cascade increases the Hill coefficient. The parameters for each reaction were chosen randomly from a beta 
distribution, denoted as a random variable z ~ B(a,/S), which yields values in the range [0, 1].The parameters m = 100zand g = 5 + 10z were 
chosen randomly and independently for each reaction from a beta distribution with a — /} — 3. The parameter k for each reaction was obtained 
randomly as 1 + z, yielding a range of coefficients 1 < k < 2. (a) In three separate trials, different combinations of (a, ft) were used for the beta 
distribution that generated the Hill coefficient, k: in the first, shown as the left distribution, (a, fi) = (1,6); in the second, shown in the middle, 
(a, P) = (4,4); in the third, on the right, (a, fS) = (6, 2). The plot shows the peak heights normalized for each curve to be the same to aid visual 
comparison. (b)The input-output relation over the full cascade. The curves from left to right correspond to the distributions for k from left to right in 
the prior panel. The input scale is normalized so that the maximum input value for each curve coincides at 80% of the maximum output that could 
be obtained at infinite input. The observed output curves have more strongly reduced sensitivity at low input than at high input compared with the 
Hill equation, but nonetheless match reasonably well. The best fit Hill equation for the three curves has a Hill coefficient of, from left to right, 
k = 1 .7, 2.2, 2.8. The average Hill coefficient for each reaction in a cascade is, from left to right, k = 1 .14, 1 .5, 1 .75. Each curve shows a single particular 
realization of the randomly chosen reaction parameters from the underlying distributions. 



retains linear sensitivity at low signal input. A Michaelis- 
Menten cascade would not have the weak sensitivity at low 
input shown in Figure 6b. 

How does a Hill coefficient k > 1 arise in the individual 
steps of the cascade? The power of aggregation to induce 
pattern means that it does not matter how such varia- 
tions in sensitivity arise. However, it is useful to consider 
some examples to gain of idea of the kinds of processes 
that may be involved beyond the deterministic cases given 
earlier. 

Signal noise versus measurement noise 

Two different kinds of noise can influence the input- 
output relations of a system. First, the initial input signal 
may be noisy, making it difficult for the system to discrimi- 
nate between low input signals and background stochastic 
fluctuations in signal intensity [43]. The classical signal- 
to-noise ratio problem expresses this difficulty by analyz- 
ing the ways in which background noise in the input can 
mask small changes in the average input intensity. When 
the signal is weak relative to background noise, a system 
may be relatively insensitive to small increases in average 
input at low input intensity. 

Second, for a given input intensity, the system may expe- 
rience noise in the detection of the signal level or in the 
transmission of the signal through the internal processes 
that determine the final output. Stochasticity in signal 
detection and transmission determine the measurement 
noise intrinsic to the system. The ratio of measurement 
noise to signal intensity will often be greater at low signal 
input intensity, because there is relatively more noise in 
the detection and transmission of weak signals. 



In this section, I consider how signal noise and measure- 
ment noise influence Michaelis-Menten processes. The 
issue concerns how much these types of noise may weaken 
sensitivity to low intensity signals. A weakening of sen- 
sitivity to low input distorts the input-output relation of 
a Michaelis-Menten process in a way that leads to a Hill 
equation type of response with k > 1. 

In terms of measurement, Michaelis-Menten processes 
follow a linear-log scaling, in which sensitivity remains 
linear and highly precise at very low signal input inten- 
sity, and grades slowly into a logarithmic scaling with 
saturation. By contrast, as the Hill coefficient, k, rises 
above one, measurement precision transforms into a log- 
linear-log scale, with weaker logarithmic sensitivity to 
signal changes at low input intensity. Thus, the prob- 
lem here concerns how signal noise or measurement 
noise may weaken input-output sensitivity at low input 
intensity. 

Input signal noise may not alter Michaelis-Menten sensitivity 

Consider the simplified Michaelis-Menten type of dynam- 
ics given in Eq. (1), repeated here for convenience 

A = gS(R -A)- SA, 

where A is the output signal, S is the input signal driving 
the reaction, R is the reactant transformed by the input, g 
is the rate of the transforming reaction which is a sort of 
signal gain level, and <5 is the rate at which the active sig- 
nal output decays to the inactive reactant form. Thus far, I 
have been analyzing this type of problem by assuming that 
the input signal, S, is a constant for any particular reaction, 
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and then varying S to analyze the input-output relation, 
given at equilibrium by Michaelis-Menten saturation 

\m + Sj 

where m = S/g. When input signal intensity is weak, such 
that m ^> S, then A* « gS, which implies that output is 
linearly related to input. 

Suppose that S is in fact a noisy input signal subject to 
random fluctuations. How do the fluctuations affect the 
input-output relation for inputs of low average intensity? 
Although the dynamics can filter noise in various ways, 
it often turns out that the linear input-output relation 
continues to hold such that, for low average input inten- 
sity, the average output is proportion to the average input, 
A oc S. Thus, signal noise does not change the fact that 
the system effectively measures the average input inten- 
sity linearly and with essentially infinite precision, even at 
an extremely low signal to noise ratio. 

The high precision of classical chemical kinetics arises 
because the mass action assumption implies that infinites- 
imal changes in input concentration are instantly trans- 
lated into a linear change in the rate of collisions between 
potential reactants. The puzzle of Michaelis-Menten 
kinetics is that mass action implies high precision and 
linear scaling at low input intensity, whereas both intu- 
ition and observation suggest low precision and logarith- 
mic scaling at low input intensity. Input signal noise by 
itself typically does not alter the high precision and linear 
scaling of mass action kinetics. 

Although the simplest Michaelis-Menten dynamics 
retain linearity and essentially infinite precision at low 
input, it remains unclear how the input-output relations 
of complex aggregate systems respond to the signal to 
noise ratio of the input. Feedback loops and reaction cas- 
cades strongly influence the ways in which fluctuations 
are filtered between input and output. However, classical 
analyses of signal processing tend to focus on the filter- 
ing properties of systems only in relation to fluctuations of 
input about a fixed mean value. By contrast, the key bio- 
logical problem is how input fluctuations alter the relation 
between the average input intensity and the average out- 
put intensity. That key problem requires one to study the 
synergistic interactions between changes in average input 
and patterns of fluctuations about the average. 

For noisy input signals, what are the universal character- 
istics of system structure and signal processing that alter 
the relations between average input and average output? 
That remains an open question. 

Noise in signal detection and transmission reduces 
measurement precision and sensitivity at low signal input 

The previous section considered how stochastic fluctu- 
ations of inputs may affect the average output. Simple 



mass action kinetics may lead to infinite precision at low 
input intensity with a linear scaling between average input 
and average output, independently of fluctuations in noisy 
inputs. This section considers the problem of noise from a 
different perspective, in which the fluctuations arise inter- 
nally to the system and alter measurement precision and 
signal transmission. 

I illustrate the key issues with a simple model. I assume 
that, in a reaction cascade with deterministic dynam- 
ics, each reaction leads to the Michaelis-Menten type of 
equilibrium input-output given in Eq. (7). To study how 
stochastic fluctuations within the system affect input- 
output relations, I assume that each reaction has a cer- 
tain probability of failing to transmit its input. In other 
words, for each reaction, the output follows the equilib- 
rium input-output relation with probability 1 —p, and with 
probability p, the output is zero. 

From the standard equilibrium in Eq. (7), we simplify the 
notation by using y = A* for output, and scale the input 
such that x = S/m. The probability that the output is not 
zero is 1 — p, thus the expected output is 

Let the probability of failure be p = ae~ bx . Note that 
as input signal intensity, x, rises, the probability of failure 
declines. As the signal becomes very small, the probability 
of reaction failure approaches a, from the range 0 < a < 1. 

Figure 7 shows that stochastic failure of signal trans- 
mission reduces relative sensitivity to low input signals 
when a signal is passed through a reaction cascade. The 
longer the cascade of reactions, the more the overall 
input-output relation follows an approximate log-linear- 
log pattern with an increasing Hill coefficient, k. Similarly, 
Figure 8 shows that an increasing failure rate per reac- 
tion reduces sensitivity to low input signals and makes the 
overall input-output relation more switch-like. 

Implications for system design 

An input-output response with a high Hill coefficient, 
k, leads to switch-like function (Figure 3). By contrast, 
classical Michaelis-Menten kinetics lead to k = 1, in 
which output increases linearly with small changes in 
weak input signals— effectively the opposite of a switch. 
Many analyses of system design focus on this distinction. 
The argument is that switch-like function will often be 
a favored feature of design, allowing a system to change 
sharply between states in response to external changes [1]. 
Because the intrinsic dynamics of chemistry are thought 
not to have a switch like function, the classical puz- 
zle is how system design overcomes chemical kinetics to 
achieve switching function. 

This section on stochastic signal failure presents an 
alternative view. Sloppy components with a tendency to 
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Figure 7 Stochastic failure of signal transmission reduces the relative sensitivity to low intensity input signals. The lower (blue) lines show 
the probability p = ae~ bx that an input signal fails to produce an output. The upper (red) lines show the expected equilibrium output for Michaelis- 
Menten type dynamics corrected for a probability p that the output is zero. Each panel (a - d) shows a cascade of n reactions, in which the output of 
each reaction forms the input for the next reaction, given an initial signal input of x for the first reaction. Each reaction follows Eq. (8). The number of 
reactions in the cascade increases from the left to the right panel as n = 1,2,4, 8. The other parameters for Eq. (8) are the gain per reaction, g = 1.5, 
the maximum probability of reaction failure as the input declines to very low intensity, a = 0.3, and the rate at which increasing signal intensity 
reduces reaction failure, b = 10. The final output signal is normalized to 0.8 of the maximum output produced as the input become very large. 



fail often lead to switch-like function. Thus, when switch- 
ing behavior is a favored phenotype, it may be sufficient 
to use a haphazardly constructed pathway of signal trans- 
mission coupled with weakly regulated reactions in each 
step. Switching, rather than being a highly designed fea- 
ture that demands a specific mechanistic explanation, may 
instead be the likely outcome of erratic biological signal 
processing. 

This tendency for aggregate systems to have a switching 
pattern does not mean that natural selection has no role 
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Figure 8 Greater failure rates for reactions reduce sensitivity to 
low input and increase the Hill coefficient, k. The curves arise from 
the same analysis as Figure 7, in which the curves from left to right are 
associated with an increase in the maximum failure rate as 
a = 0.2, 0.4, 0.6. The curves in this figure have n = 8 reactions in the 
cascade, a gain of g = 1.5, and a decline in failure with increasing 
input, b = 10. The scale for the input signal is normalized so that each 
curve has a final output of 0.85 at a normalized input of one. 



and that system design is random. Instead, the correct 
view may be that aggregate signal processing and inherent 
stochasticity set the contours of variation on which nat- 
ural selection and system design work. In particular, the 
key design features may have to do with modulating the 
degree of sloppiness or stochasticity. The distribution of 
gain coefficients in each reaction and the overall pattern 
of stochasticity in the aggregate may also be key loci of 
design. 

My argument is that systems may be highly designed, 
but the nature of that design can only be understood 
within the context of the natural patterns of variation. The 
intrinsic contours of variation are the heart of the matter. 
I will discuss that issue again later. For now, I will con- 
tinue to explore the processes that influence the nature of 
variation in system input-output patterns. 

Spatial correlations and departures from mass action 

Chemical reactions require molecules to come near each 
other spatially. The overall reaction depends on the pro- 
cesses that determine spatial proximity and the processes 
that determine reaction rate given spatial proximity. 
Roughly speaking, we can think of the spatial aspects in 
terms of movement or diffusion, and the transformation 
given spatial proximity in terms of a reaction coefficient. 

Classical chemical kinetics typically assumes that dif- 
fusion rates are relatively high, so that spatial proximity 
of molecules depends only on the average concentration 
over distances much greater than the proximity required 
for reaction. Kinetics are therefore limited by reaction rate 
given spatial proximity rather than by diffusion rate. In 
contrast with classical chemical kinetics, much evidence 
suggests that biological molecules often diffuse relatively 
slowly, causing biological reactions sometimes to be diffu- 
sion limited (Table 2). 
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In this section, I discuss how diffusion-limited reactions 
can increase the Hill coefficient of chemical reactions, 
k > 1. That conclusion means that the inevitable limi- 
tations on the movement of biological molecules may be 
sufficient to explain the observed patterns of sensitivity 
in input-output functions and departure from Michaelis- 
Menten patterns. 

Two key points emerge. First, limited diffusion tends 
to cause potential reactants to become more spatially 
separated than expected under high diffusion and ran- 
dom spatial distribution. The negative spatial associa- 
tion between reactants arises because those potential 
reactants near each tend to react, leaving the nearby 
spatial neighborhood with fewer potential reactants 
than expected under spatial uniformity. Negative spa- 
tial association of reactants reduces the rate of chemical 
transformation. 

This reduction in transformation rate is stronger at 
low concentration, because low concentration is associ- 
ated with a greater average spatial separation of reactants. 
Thus, low signal input may lead to relatively strong reduc- 
tions in transformation rate caused by limited diffusion. 
As signal intensity and concentration rise, this spatial 
effect is reduced. The net consequence is a low trans- 
formation rate at low input, with rising transformation 
rate as input intensity increases. This process leads to the 
the pattern characterized by higher Hill coefficients and 
switch-like function, in which there is low sensitivity to 
input at low signal intensity. 

Limited diffusion within the broader context of input- 
output patterns leads to the second key point. I will 
suggest that limited diffusion is simply another way in 
which systems suffer reduced measurement precision 
and loss of information at low signal intensity. The ulti- 
mate understanding of system design and input-output 
function follows from understanding how to relate par- 
ticular mechanisms, such as diffusion or random sig- 
nal loss, to the broader problems of measurement and 
information. To understand those broader and more 
abstract concepts of measurement and information, it 
is necessary to work through some of the particular 
details by which diffusion limitation leads to loss of 
information. 

Departure from mass action 

Most analyses of chemical kinetics assume mass action. 
Suppose, for example, that two molecules may combine to 
produce a bound complex 

A + B 4 AB 

in which the bound complex, AB, may undergo further 
transformation. Mass action assumes that the rate at 
which AB forms is rAB, which is the product of the concen- 
trations of A and B multiplied by a binding coefficient, r. 



The idea is that the number of collisions and potential 
binding reactions between A and B per unit of time 
changes linearly with the concentration of each reactant. 

Each individual reaction happens at a particular loca- 
tion. That particular reaction perturbs the spatial asso- 
ciation between reactants. Those reactants that were, by 
chance, near each other, no longer exist as free poten- 
tial reactants. Thus, a reaction reduces the probability 
of finding potential reactants nearby, inducing a negative 
spatial association between potential reactants. To retain 
the mass action rate, diffusion must happen sufficiently 
fast to break down the spatial association. Fast diffusion 
recreates the mutually uniform spatial concentrations of 
the reactants required for mass action to hold. 

If diffusion is sufficiently slow, the negative spatial asso- 
ciation between reactants tends to increase over time as 
the reaction proceeds. That decrease in the proximity 
of potential reactants reduces the overall reaction rate. 
Diffusion-limited reactions therefore have a tendency for 
the reaction rate to decline below the expected mass 
action rate as the reaction proceeds. 

That classical description of diffusion-limited reactions 
emphasizes the pattern of reaction rates over time. By 
contrast, my focus is on the relation between input and 
output. It seems plausible that diffusion limitation could 
affect the input-output pattern of a biological system. But 
exactly how should we connect the classical analysis of dif- 
fusion limitation for the reaction rate of simple isolated 
reactions to the overall input-output pattern of biological 
systems? 

The connection between diffusion and system input- 
output patterns has received relatively little attention. A 
few isolated studies have analyzed the ways in which dif- 
fusion limitation tends to increase the Hill coefficient, 
supporting my main line of discussion (Table 2). However, 
the broad field of biochemical and cellular responses has 
almost entirely ignored this issue. The following sections 
present a simple illustration of how diffusion limitation 
may influence input-output patterns, and how that effect 
fits into the broader context of the subject. 

Example of input-output pattern under limited diffusion 

Limited diffusion causes spatial associations between 
reactants. Spatial associations invalidate mass action 
assumptions. To calculate reaction kinetics without mass 
action, one must account for spatially varying concen- 
trations of reactants and the related spatial variations in 
chemical transformations. There is no simple and gen- 
eral way to make spatially explicit calculations. In some 
cases, simple approximations give a rough idea of out- 
come (Table 2). However, in most cases, one must study 
reaction kinetics by spatially explicit computer simula- 
tions. Such simulations keep track of the spatial location 
of each molecule, the rate at which nearby molecules 
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react, the spatial location of the reaction products, and the 
stochastic movement of each molecule by diffusion. 

Many computer packages have been developed to aid 
stochastic simulation of spatially explicit biochemical 
dynamics. I used the package Smoldyn [33,44]. I focused 
on the ways in which limited diffusion may increase Hill 
coefficients. Under classical assumptions about chemi- 
cal kinetics, diffusion rates tend to be sufficiently high to 
maintain spatial uniformity, leading to Michaelis-Menten 
kinetics with a Hill coefficient of k = 1. With lower dif- 
fusion rates, spatial associations arise, invalidating mass 
action. Could such spatial associations lead to increased 
Hill coefficients of k > 1? 

Figure 9 shows clearly that increased Hill coefficients 
arise readily in a simple reaction scheme with limited 
diffusion. The particular reaction system is 



S+R4S+A 
X + A A X + R. 



(9) 
(10) 



Under mass action assumptions, the dynamics would be 
identical to Eq. (1) 

A = gS(N -A)- SXA, 

in which N = R + A is the total concentration of inactive 
plus active reactant molecules and, in this case, we write 
the back reaction rate as SX rather than just <5 as in the ear- 
lier equation. In a spatially explicit model, we must keep 
track of the actual spatial location of each X molecule, 
thus we need to include explicitly the concentration X 
rather than include that concentration in a combined 
rate parameter. At equilibrium, the output signal intensity 
under mass action follows the Michaelis-Menten relation 



N 



S 



in which m = SX/g. If we let x = S/m and y = A* /N, 
then we see that the reaction scheme here leads to an equi- 
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Figure 9 Limited diffusion and spatial association of reactants can increase the Hill coefficient, k. Simulations shown from the computer 
package Smoldyn, based on the reaction scheme in Eqs. (9,10). The concentration of the input signal, 5, is the number of molecules per unit volume. 
The other concentrations are set to N = X = 1 00. Diffusion rates are 1 0~ 5 for all molecules. I ran three replicates for each input concentration, 5. 
Each circle shows the average of the three replicates. For each panel (a - f), I fit a Hill equation curve to the observations, denoting the output as the 
relative saturation level, A/N = sat [S / (m k + 5*)]. The fitted parameters are: k, the Hill coefficient; m, the input signal concentration that yields 
one-half of maximum saturation; and "sat", the maximum saturation level at which the output is estimated to approach an asymptotic value relative 
to the maximum theoretical value of one, at which all N has been transformed into A. Because of limited diffusion, actual saturation can be below 
the theoretical maximum of one. Panels (b) and (c) are limited to output responses far below the median, because the simulations take too long to 
run for higher input concentrations. 
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librium input-output relation as in Eq. (4) that follows the 
Hill equation 




with k = 1. 

I used the Smoldyn simulation package to study reac- 
tion dynamics when the mass action assumption does not 
hold. The simulations for this particular reaction scheme 
show input-output relations with k > 1 when the rates of 
chemical transformation are limited by diffusion. Figure 9 
summarizes some Smoldyn computer simulations show- 
ing k significantly greater than one for certain parameter 
combinations. I will not go into great detail about these 
computer simulations, which can be rather complicated. 
Instead, I will briefly summarize a few key points, because 
my goal here is simply to illustrate that limited diffu- 
sion can increase Hill coefficients under some reasonable 
conditions. 

It is clear from Figure 9 that limited diffusion can raise 
the Hill coefficient significantly above one. What causes 
the rise? It must be some aspect of spatial process, because 
diffusion limitation primarily causes departure from mass 
action by violating the assumption of spatial uniformity. I 
am not certain which aspects of spatial process caused the 
departures in Figure 9. It appeared that, in certain cases, 
most of the transformed output molecules, A, were main- 
tained in miniature reaction centers, which spontaneously 
formed and decayed. 

A local reaction center arose when S and R molecules 
came near each other, transforming into S and A. If there 
was also a nearby X molecule, then X and A caused a 
reversion to X and R. The R molecule could react again 
with the original nearby S molecule, which had not moved 
much because of a slow diffusion rate relative to the 
timescale of reaction. The cycle could then repeat. If for- 
mation of reaction centers rises nonlinearly with signal 
concentration, then a Hill coefficient k > 1 would follow. 

Other spatial processes probably also had important, 
perhaps dominant, roles, but the miniature reaction cen- 
ters were the easiest to notice. In any case, the spa- 
tial fluctuations in concentration caused a significant 
increase in the Hill coefficient, k, for certain parameter 
combinations. 

Limited diffusion, measurement precision and information 

Why do departures from spatial uniformity and mass 
action sometimes increase the Hill coefficient? Roughly 
speaking, one may think of the inactive reactant, R, as 
a device to measure the signal input concentration, S. 
The rate of SR binding is the informative measurement. 
The measurement scale is linear under spatial uniformity 
and mass action. The measurement precision is essentially 



perfect, because SR complexes form at a rate exactly lin- 
early related to S, no matter how low the concentration S 
may be and for any concentration R. 

Put another way, mass action implies infinite linear mea- 
surement precision, even at the tiniest signal intensities. 
By contrast, with limited diffusion and spatial fluctua- 
tions in concentration, measurement precision changes 
with the scale of the input signal intensity. For example, 
imagine a low concentration input signal, with only a few 
molecules in a local volume. An SR binding transforms R 
into A, reducing the local measurement capacity, because 
it is the R molecules that provide measurement. 

With slow diffusion, each measurement alters the 
immediate capacity for further measurement. The 
increase in information from measurement is partly offset 
by the loss in measurement capacity. Put another way, the 
spatial disparity in the concentration of the measuring 
device R is a loss of entropy, which is a sort of gain in 
unrealized potential information. As unrealized potential 
information builds in the spatial disparity of R, the capac- 
ity for measurement and the accumulation of information 
about S declines, perhaps reflecting a conservation princi- 
ple for total information or, equivalently, for total entropy 
at steady state. 

At low signal concentration, each measurement reaction 
significantly alters the spatial distribution of molecules 
and the measurement capacity. As signal concentration 
rises, individual reactions have less overall effect on spa- 
tial disparity. Put another way, the spatial disparities 
increase as signal intensity declines, causing measurement 
to depend on scale in a manner that often leads to a log- 
arithmic scaling. I return to the problem of logarithmic 
scaling below. 

Shaping sensitivity and dynamic range 

The previous sections considered specific mechanisms 
that may alter sensitivity of input-output relations in 
ways that lead to the log-linear-log scaling of the Hill 
equation. Such mechanisms include stochastic failure of 
signal processing in a cascade or departures from mass 
action. Those mechanisms may be important in many 
cases. However, my main argument emphasizes that the 
widespread occurrence of log-linear-log scaling for input- 
output relations must transcend any particular mecha- 
nism. Instead, general properties of system architecture, 
measurement and information flow most likely explain 
the simple regularity of input-output relations. Those gen- 
eral properties, which operate at the system level, tend to 
smooth out the inevitable departures from regularity that 
must occur at smaller scales. 

Brief review and setup of the general problem 

An increase in the Hill coefficient, k, reduces sensitivity 
at low and high input signal intensity (Figure 2). At those 
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intensities, small changes in input cause little change in 
output. Weak sensitivity tends to be logarithmic, in the 
sense that output changes logarithmically with input. Log- 
arithmic sensitivities at low and high input often cause 
sensitivity to be strong and nearly linear within an inter- 
mediate signal range, with a rapid rate of change in output 
with respect to small changes in input intensity. The inter- 
mediate interval over which high sensitivity occurs is the 
dynamic range. The Hill coefficient often provides a good 
summary of the input-output pattern and is therefore a 
useful method for studying sensitivity and dynamic range. 

The general problem of understanding biological input- 
output systems can be described by a simple question. 
What processes shape the patterns of sensitivity and 
dynamic range in biological systems? To analyze sensitiv- 
ity and dynamic range, we must consider the architecture 
by which biological systems transform inputs to outputs. 

Aggregation of multiple transformations 

Biological systems typically process input signals through 
numerous transformations before producing an output 
signal. Thus, the overall input-output pattern arises from 
the aggregate of the individual transformations. Although 
the meaning of "output signal" depends on context, mean- 
ingful outputs typically arise from multiple transforma- 
tions of the original input. 

I analyzed a simple linear cascade of transformations in 
an earlier section. In that case, the first step in the cascade 
transforms the original input to an output, which in turn 
forms the input for the next step, and so on. If individ- 
ual transformations in the cascade have Hill coefficients 
k > 1, the cascade tends to amplify the aggregate coef- 
ficient for the overall input-output pattern of the system. 
Amplification occurs because weak logarithmic sensitivi- 
ties at low and high inputs tend to multiply through the 
cascade. Multiplication of logarithmic sensitivities at the 
outer ranges of the signal raises the overall Hill coefficient, 
narrows the dynamic range, and leads to high sensitivity 
over intermediate inputs. 

That amplification of Hill coefficients in cascades leads 
back to the puzzle I have emphasized throughout this 
article. For simple chemical reactions, kinetics follow the 
Michaelis-Menten pattern with a Hill coefficient of k = 1. 
If classical kinetics are typical, then aggregate input- 
output relations should also have Hill coefficients near 
to one. By contrast, most observed input-output pat- 
terns have higher Hill coefficients. Thus, some aspect of 
the internal processing steps must depart from classical 
Michaelis-Menten kinetics. 

There is a long history of study with regard to the mech- 
anisms that lead individual chemical reactions to have 
increased Hill coefficients. In the first part of this arti- 
cle, I summarized three commonly cited mechanisms of 
chemical kinetics that could raise the Hill coefficient for 



individual reactions: cooperative binding, titration of a 
repressor, and opposing saturated forward and back reac- 
tions. Those sorts of deterministic mechanisms of chem- 
ical kinetics do raise Hill coefficients and probably occur 
in many cases. However, the generality of raised Hill coef- 
ficients seems to be too broad to be explained by such 
specific deterministic mechanisms. 

Component failure 

If the classical deterministic mechanisms of chemical 
kinetics do not sufficiently explain the generality of raised 
Hill coefficients, then what does explain that general- 
ity? My main argument is that input-output relations 
reflect underlying processes of measurement and infor- 
mation. The nature of measurement and information 
leads almost inevitably to the log-linear-log pattern of 
observed input-output relations. That argument is, how- 
ever, rather abstract. How do we connect the abstractions 
of measurement and information to the actual chemical 
processes by which biological systems transform inputs to 
outputs? 

To develop the connection between abstract concepts 
and underlying mechanisms of chemical kinetics, I pre- 
sented a series of examples. I have already discussed 
aggregation, perhaps the most powerful and important 
general concept. I showed that aggregation amplifies small 
departures from Michaelis-Menten kinetics (k = 1) into 
strongly log-linear-log patterns with increased k. 

In my next step, I showed that when individual com- 
ponents of an aggregate system have Michaelis-Menten 
kinetics but also randomly fail to transmit signals with 
a certain probability, the system converges on an input- 
output pattern with a raised Hill coefficient. The main 
assumption is that failure rate increases as signal input 
intensity falls. 

Certainly, some reactions in biological systems will tend 
to fail occasionally, and some of those failures will be cor- 
related with input intensity. Thus, a small and inevitable 
amount of sloppiness in component performance of an 
aggregate system alters the nature of input-output mea- 
surement and information transmission. Because the con- 
sequence of failures tends to multiply through a cascade, 
logarithmic sensitivity at low signal input intensity follows 
inevitably. 

Rather than invoke a few specific chemical mecha- 
nisms to explain the universality of log-linear-log scaling, 
this view invokes the universality of aggregate processing 
and occasional component failures. I am not saying that 
component failures are necessarily the primary cause of 
log-linear-log scaling. Rather, I am pointing out that such 
universal aspects must be common and lead inevitably 
to certain patterns of measurement and information pro- 
cessing. Once one begins to view the problem in this way, 
other aspects begin to fall into place. 
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Departure from mass action 

Limited rates of chemical diffusion often occur in biolog- 
ical systems. I showed that limited diffusion may distort 
classical Michaelis-Menten kinetics to raise the Hill coef- 
ficient above one. The increased Hill coefficient, and 
associated logarithmic sensitivity at low input, may be 
interpreted as reduced measurement precision for weak 
signals. 

Regular pattern from highly disordered mechanisms 

The overall conclusion is that many different mechanisms 
lead to the same log-linear-log scaling. In any particular 
case, the pattern may be shaped by the classical mech- 
anisms of binding cooperativity, repressor titration, or 
opposing forward and back reactions. Or the pattern may 
arise from the generic processes of aggregation, compo- 
nent failure, or departures from mass action. 

No particular mechanism necessarily associates with 
log-linear-log scaling. Rather, a broader view of the rela- 
tions between pattern and process may help. That broader 
view emphasizes the underlying aspects of measurement 
and information common to all mechanisms. The com- 
mon tendency for input-output to follow log-linear-log 
scaling may arise from the fact that so many different 
processes have the same consequences for measurement, 
scaling and information. 

The common patterns of nature are exactly those pat- 
terns consistent with the widest, most disparate range 
of particular mechanisms. When great underlying disor- 
der has, in the aggregate, a rigid common outcome, then 
that outcome will be widely observed, as if the outcome 
were a deterministic inevitability of some single underly- 
ing cause. The true underlying cause arises from generic 
aspects of measurement and information, not with spe- 
cific chemical mechanisms. 

System design 

The inevitability of log-linear-log scaling from diverse 
underlying mechanisms suggests that the overall shape 
of biological input-output relations may be strongly con- 
strained. Put another way, the range of variation is limited 
by the tendency to converge to log-linear-log scaling. 
However, within that broad class of scaling, biological sys- 
tems can tune the responses in many different ways. The 
tuning may arise by adjusting the number of reactions in a 
cascade, by allowing component failure rates to increase, 
by using reactions significantly limited by diffusion rate, 
and so on. 

Understanding the design of input-output relations 
must focus on those sorts of tunings within the broader 
scope of measurement and information transmission. The 
demonstration that a particular mechanism occurs in a 
particular system is always interesting and always lim- 
ited in consequence. The locus of design and function 



is not the particular mechanism of a particular reaction, 
but the aggregate properties that arise through the many 
mechanisms that influence the tuning of the system. 

Robustness 

Overall input-output pattern often reflects the tight order 
that arises from underlying disorder. Thus, perturbations 
of particular mechanisms in the system may often have 
relatively little consequence for overall system function. 
That insensitivity to perturbation — or robustness — arises 
naturally from the structure of signal processing in biolog- 
ical systems. 

To study robustness, it may not be sufficient to search 
for particular mechanisms that reduce sensitivity to per- 
turbation. Rather, one must understand the aggregate 
nature of variation and function, and how that aggregate 
nature shapes the inherent tendency toward insensitivity 
in systems [3,4,45]. Once one understands the intrinsic 
properties of biological systems, then one can ask how 
those intrinsic properties are tuned by natural selection. 

Measurement and information 

Intuitively, it makes sense to consider input-output rela- 
tions with respect to measurement and information. How- 
ever, one may ask whether "measurement" and "informa- 
tion" are truly useful concepts or just vague and ultimately 
useless labels with respect to analyzing biological sys- 
tems. Here, I make the case that deep and useful concepts 
underlie "measurement" and "information" in ways that 
inform the study of biological design (Table 1). I start by 
developing the abstract concepts in a more explicit way. I 
then connect those abstractions to the nature of biological 
input-output relations. 

Measurement 

Measurement is the assignment of a value to some under- 
lying attribute or event. Thus, we may think of input- 
output relations in biology as measurement relations. At 
first glance, this emphasis on measurement may seem 
trivial. What do we gain by thinking of every chemical 
reaction, perception, or dose-response curve as a process 
of measurement? 

Measurement helps to explain why certain similarities 
in pattern continually arise. When we observe common 
patterns, we are faced a question. Do common aspects 
of pattern between different systems arise from univer- 
sal aspects of measurement or from particular mech- 
anisms of chemistry or perception shared by different 
systems? 

Problems arise if we do not think about the distinction 
between general properties of measurement and specific 
mechanisms of particular chemical pathways. If we do 
not think about that distinction, we may try to explain 
what is in fact a universal attribute of measurement by 
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searching, in each particular system, for special aspects of 
chemical kinetics, pathway structure or physical laws that 
constrain perception. In the opposite direction, we can 
never truly recognize the role of particular mechanisms 
in generating observed patterns if we do not separate 
out those aspects of pattern that arise from universal 
process. 

Understanding universal aspects of pattern that arise 
from measurement means more than simply analyzing 
how observations are turned into numbers. Instead, we 
must recognize that the structure of each problem sets 
very strong constraints on numerical pattern indepen- 
dently of particular chemical or biological mechanisms. 

Log-linear-log scales 

I have mentioned that the Hill equation is simply an 
expression of log-linear-log scaling. The widely recog- 
nized value of the Hill equation for describing biological 
pattern arises from its connection to that underlying uni- 
versal scale of measurement, in which small magnitudes 
scale logarithmically, intermediate magnitudes scale lin- 
early, and large values scale logarithmically. Although 
linear and logarithmic scales are widely used and very 
familiar, the actual properties and meanings of such scales 
are rarely discussed. If we consider directly the nature 
of measurement scale, we can understand more deeply 
how to understand the relations between pattern and 
process. 

Consider the example of measuring distance [41]. Start 
with a ruler that is about the length of your hand. With 
that ruler, you can measure the size of all the visible 
objects in your office. That scaling of objects in your 
office with the length of the ruler means that those 
objects have a natural linear scaling in relation to your 
ruler. 

Now consider the distances from your office to various 
galaxies. Your ruler is of no use, because you cannot dis- 
tinguish whether a particular galaxy moves farther away 
by one ruler unit. Instead, for two galaxies, you can mea- 
sure the ratio of distances from your office to each galaxy. 
You might, for example, find that one galaxy is twice as far 
as another, or, in general, that a galaxy is some percent- 
age farther away than another. Percentage changes define 
a ratio scale of measure, which has natural units in loga- 
rithmic measure [5]. For example, a doubling of distance 
always adds log (2) to the logarithm of the distance, no 
matter what the initial distance. 

Measurement naturally grades from linear at local 
magnitudes to logarithmic at distant magnitudes when 
compared to some local reference scale. The transition 
between linear and logarithmic varies between problems. 
Measures from some phenomena remain primarily in the 
linear domain, such as measures of height and weight in 
humans. Measures for other phenomena remain primarily 



in the logarithmic domain, such as cosmological dis- 
tances. Other phenomena scale between the linear and 
logarithmic domains, such as fluctuations in the price of 
financial assets [46] or the distribution of income and 
wealth [47]. 

Consider the opposite direction of scaling, from local 
magnitude to very small magnitude. Your hand-length 
ruler is of no value for small magnitudes, because it can- 
not distinguish between a distance that is a fraction 10~ 4 
of the ruler and a distance that is 2 x 10~ 4 of the ruler. At 
small distances, one needs a standard unit of measure that 
is the same order of magnitude as the distinctions to be 
made. A rule of length 10 -4 distinguishes between 10 -4 
and 2 x 10 -4 , but does not distinguish between 10 -8 and 
2 x 10 -8 . At small magnitudes, ratios can potentially be 
distinguished, causing the unit of informative measure to 
change with scale. Thus, small magnitudes naturally have 
a logarithmic scaling. 

As we change from very small to intermediate to very 
large, the measurement scaling naturally grades from log- 
arithmic to linear and then again to logarithmic, a log- 
linear-log scaling. The locus of linearity and the meaning 
of very small and very large differ between problems, 
but the overall pattern of the scaling relations remains 
the same. This section analyzes that characteristic scal- 
ing in relation to the Hill equation and biological input- 
output patterns. I start by considering more carefully what 
measurement scales mean. I then connect the abstract 
aspects of measurement to the particular aspects of the 
Hill equation and to examples of particular biological 
mechanisms. 

Invariance, the essence of explanation 

We began with an observation. Many different input- 
output relations follow the Hill equation. We then asked: 
What process causes the Hill equation pattern? It turned 
out that many very different kinds of process lead to the 
same log-linear-log pattern of the Hill equation. We must 
change our question. What do the very different kinds of 
process have in common such that they generate the same 
overall pattern? 

Consider two specific processes discussed earlier, coop- 
erative binding and departures from mass action. Those 
different processes may produce Hill equation patterns 
with similar Hill coefficients, k. However, it is not imme- 
diately obvious why cooperative binding, departures from 
mass action, and many other different processes should 
lead to a very similar pattern. 

Group together all of the disparate mechanisms that 
generate a common Hill equation pattern. When faced 
with a new mechanism, how can we tell if it belongs to the 
group? We might look for particular features that are com- 
mon to all members of the group. However, that does not 
work. Various potential members might have important 
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common features. But the attributes that they do not share 
might cause one potential member to have a different 
pattern. Common features are not sufficient. 

More often common membership arises from the fea- 
tures that do not matter. Think of circles. How can we 
describe whether a shape belongs to the circle class? We 
have to say what does not matter. For circles, it does not 
matter how much one rotates them, they always look the 
same. Circles are invariant to any rotation. Equivalently, 
circles are symmetric with regard to any rotation. Invari- 
ance and symmetry are the same thing. Subject to some 
constraints, if a shape is invariant to any rotation, it is a 
circle. If it is not invariant to all rotations, it is not a cir- 
cle. The things that do not matter set the shared, invariant 
property of a group [48-50]. 

A rotation is a kind of transformation. The group 
is defined by the set of transformations that leave the 
group members unchanged, or invariant. We can alter 
a chemical system from cooperative binding under mass 
action to noncooperative binding under departure from 
mass action, and the log-linear-log scaling may be pre- 
served. Such invariance arises because the different pro- 
cesses have an underlying symmetry with regard to the 
transformation of information from inputs to outputs 
(Table 1). 

What aspects of process do not matter with respect 
to causing the same log-linear-log pattern of the Hill 
equation? How can we recognize the underlying invari- 
ance that joins together such disparate processes with 
respect to common pattern? The Hill equation expresses 
measurement scale. To answer our key questions, we must 
understand the meaning of measurement scale. Measure- 
ment scale itself is solely an expression of invariance. A 
particular measurement scale expresses what does not 
matter— the invariance under transformation that joins 
different kinds of processes to a common scaling. 

Invariance and measurement 

Suppose a process transforms inputs x to outputs G(x). 
The process may be a reading from a measurement instru- 
ment or a series of chemical transformations. Given that 
process, how should we define the associated measure- 
ment scale? Definitions can, of course, be made in any 
way. But we should aim for something with reasonable 
meaning. 

One possible meaning for measurement is the scale that 
preserves information. In particular, we seek a scale on 
which we obtain the same information from the values of 
the inputs as we do from the values of the outputs. The 
measurement scale is the scale on which the input-output 
transformation does not alter the information in the signal 
(Table 1). 

Information is, of course, often lost between input and 
output. But only certain kinds of information are lost. 



The measurement scale describes exactly what sort of 
information is lost during the transformation from input 
to output and what sort of information is retained. In 
other words, the measurement scale defines the invari- 
ant qualities of information that remain unchanged by the 
input-output process. 

Different input-output processes belong to the same 
measurement scale when they share the same invariance 
that leaves particular aspects of information unchanged. 
For such processes, certain aspects of information remain 
the same whether we have access to the original inputs 
or the final outputs when those values are given on 
the associated measurement scale. By contrast, input- 
output processes that alter those same aspects of infor- 
mation when input and output values are given by a 
particular measurement scale do not belong to that 
scale. 

Those abstract properties define a reasonable meaning 
for measurement scale. Such abstractness can be hard to 
parse. However, it is essential to have a clear expression 
of those ideas, otherwise we could never understand why 
so many different kinds of biological processes can have 
such similar input-output relations, and why other pro- 
cesses do not share the same relations. It is exactly those 
abstract informational aspects of measurement that unite 
cooperative binding and departures from mass action into 
a common group of processes that share a similar Hill 
equation pattern. 

Measurement and information 

It is useful to express the general concepts in a simple 
equation. I build up to that simple summary equation by 
starting with components of the overall concept. 

Inputs are given by x. We denote a small change in input 
by d*. An input given on the measurement scale is T(x). 
The sensitivity of the measurement scale to a change in 
input is 

dT(x) 
m x = — — , 
d* 

which is the change on the measurement scale, dJ(x), 
with respect to a change in input, d*. That sensitivity 
describes the information in the measurement scale with 
respect to fluctuations in inputs [41,42,51]. We may also 
write 

m x dx = dT(x), 

providing an expression for the incremental information 
associated with a change in the underlying input, dx. If the 
scale is logarithmic, T(x) = log(#), then 

dx 

m x dx = dlog(*) = — , 

x 
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for which the sensitivity of the measurement scale 
declines as the input becomes large. On a purely logarith- 
mic scale, the same increment in input, dx, provides a lot 
of information when x is small and little information when 
x is large. 

Next, we express the relation that defines measurement 
scale. On the proper measurement scale for a particular 
problem, the information from input values is propor- 
tional to the information from associated output values. 
Put another way, the measurement scale is the trans- 
formation of values that makes information invariant to 
whether we use the input values or the output values. 
The measurement scale reflects those aspects of infor- 
mation that are preserved in the input-output relation, 
and consequently also expresses those aspects of informa- 
tion that are lost in the input-output relation. Although 
rather abstract, it is useful to complete the mathematical 
development before turning to some examples in the next 
section. 

The output is G(x), and the measurement scale trans- 
forms the output by T [G(#)]. To have proportionality for 
the incremental information associated with a change in 
the underlying input, dT(x), and the incremental infor- 
mation associated with a change in the associated output, 
dT [G(X>], we have 

dT(x) oc dT [G(*)] (11) 

in which the oc relationship shows the proportionality 
of information associated with the sensitivity of inputs 
and outputs when expressed on the measurement scale. 
That measurement scale defines the group of input-output 
processes, G(x), that preserves the same invariant sen- 
sitivity and information properties on the scale T(x). In 
other words, all such input-output processes G(x) that are 
invariant to the measurement scale transformation T(x) 
belong to that measurement scale [41,42,51]. 

In this equation, we have inputs, x, with the informa- 
tion in those inputs, dT(x), on the measurement scale 
T, and outputs, G(x), with information in those outputs, 
dT [G(X)], on the measurement scale T. We may abbre- 
viate this key equation of measurement and information 
as 

dT cx dT [G] 

which we read as the information in inputs, dT, is pro- 
portional to the information in outputs, dT [G]. All input- 
output relations G(x) that satisfy this relation have the 
same invariant informational properties with respect to 
the measurement scale T. 

Linear scale 

This view of measurement scale means that linearity has 
an exact definition. Linearity requires that we obtain the 
same information from an increment d* on the input scale 



independently of whether the actual value is big or small 
(location), and whether we uniformly stretch or shrink 
all measurements by a constant amount. To expresses 
changes in location and in uniform scaling, let 

T(x) = a + bx, 

which changes the initial value, x, by altering the location 
by a and the uniform stretching or shrinking (scaling) by 
b. This transformation is often called the linear transfor- 
mation. But why is that the essence of linearity? From the 
first part of Eq. (11) 

m x dx = dT(x) = b dx oc dx, 

which means that an increment in measurement pro- 
vides a constant amount of information no matter what 
the measurement value, and that the information is uni- 
form apart from a constant of proportionality b. Linearity 
means that information in measurements is independent 
of location and uniform scaling. 

What sort of input-output relations, G(x), belong to the 
linear measurement scale? From the second part of Eq. 
(11), we have dT [G(#)] oc dx, which we may expand as 

dT [G(x)] = d[a + bG(x)] 
= bdG(x) oc dx 

Thus, any input-output relations such that dG(x) oc dx 
belong to the linear scale, and any input-output relations 
that do not satisfy that condition do not belong to the 
linear scale. To satisfy that condition, the input-output 
relation must have the form G(x) = a + fix, which 
is itself a linear transformation. So, only linear input- 
output relations attach to a linear measurement scale. If 
the input-output relation is not linear, then the proper 
measurement scale is not linear. 

Logarithmic scale 

We can run the same procedure on the logarithmic mea- 
surement scale, for which a simple form is T(x) = log(#). 
For this scale, dT(x) = dx/x. Thus, input-output relations 
belong to this logarithmic scale if 

dT [G(x)] = dlog [G(*)] 

dG(x) dx 
G(x) x 

This condition requires that G{x) oc x k , for which 
dG(x) oc x k ~ l dx. The logarithmic measurement scale 
applies only to input-output functions that have this 
power-law form (Table 1). Note that the special case of 
k = 1 leads to linear scaling, but for other k values the 
scale is logarithmic. 

Linear-log and log-linear scales 

The most commonly used measurement scales are linear 
and logarithmic. But those scales are unnatural, because 
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the properties of measurement likely change with mag- 
nitude. As I mentioned earlier, an office ruler is fine for 
making linear measurements on the visible objects in your 
office. But if you scale up to cosmological distances or 
down in microscopic distances, you naturally grade from 
linear to logarithmic. A proper sense of measurement 
requires attention to the ways in which information and 
input-output relations change with magnitude [41,42]. 
Suppose an input increment provides information as 

dx 

m x dx ■■ 



1 + bx 

When x is small, m x dx « dx, which is the linear mea- 
surement scale. When x is large, m x dx ^ dx/x, which is 
the logarithmic scale. The associated measurement scale 
is 

T(x) oc log(l + bx), 

and the associated input-output functions satisfy G(x) oc 
(1 + bx) k . This scale grades continuously from linear 
to logarithmic. The parameter b determines the relation 
between magnitude and the type of scaling. 

The inverse scaling grades from logarithmic at small 
magnitudes to linear as magnitude increases, with 

T(x) oc x + blog(x). 

When x is small, the scale is logarithmic with T(x) «s 
b \og(x). When x is large, the scale is linear with T(x) «a x. 



Whence is small, dT(x) oc dx/x, the expression for input- 
output functions associated with the logarithmic scale. 
When x is large, dT(^c) oc — dx/x, which is the expression 
for saturation on a logarithmic scale. 

When k > 1, the input-output relation scales linearly 
for intermediate x values. One can do various calculations 
to show the approximate linearity in the middle range. But 
the main point can be seen by simply looking at Figure 2. 

Exact linearity occurs when the second derivative of the 
Hill equation vanishes at 



1 



1 



1//C 



(12) 



for k > 1. Figure 10 shows that the locus of linearity shifts 
from the low side as k — > 1 and x* — > 0 to the high side 
as k — > oo and x* — > 1. Note that x* = 1 is the input at 
which the response is one-half of the maximum. 

Sensitivity and information 

Sensitivity is the responsiveness of output for a small 
change in input. For a log-linear-log pattern, the locus of 
linearity is often equivalent to maximum sensitivity of the 
output in relation to the input. The logarithmic regimes 
at low and high input are relatively weakly sensitive to 
changes in input. 
The Hill equation pattern for input, x, and output, G(x), 



Biological input-output: log-linear-log 

I have emphasized that the log-linear-log scale is perhaps 
the most natural of all scales. Information in measurement 
increments tends to be logarithmic at small and large 
magnitudes. As one moves in either extreme direction, 
the unit of measure changes in proportion to magni- 
tude to preserve consistent information. At intermediate 
magnitudes, changing values associate with an approx- 
imately linear measurement scale. For many biological 
input-output relations, that intermediate, linear zone is 
roughly the dynamic range. 
The Hill equation description of input-output relations 

x k 

G(X) = ——r, 



G(x) = 



is widely useful because it describes log-linear-log scaling 
in a simple form. To check for log scaling in the lim- 
its of high or low input, we use T(x) = \og(x), which 
implies dT(x) oc dx/x. In our fundamental relation of 
measurement, we have 

dT(*) ocdT [G(x)] 
= dlog[G(*)] 



1 



d*, 



1 



The equivalent form on the right side is the classic logis- 
tic function expressed in terms of log(^) rather than x. 
This logarithmic form is the log-logistic function. Note 
also that G(*) varies between zero and one as x increases 




2 3 

Hill coefficient, k 

Figure 10 The locus of linearity, which is the value of input, a*, at 
which the log-linear-log pattern of the Hill equation becomes 
exactly linear. The locus of linearity corresponds to the peak 
sensitivity of the input-output relation. At x* = 1, output is one-half 
of maximal response. Plot based on Eq. (1 2). 
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from zero. Thus, G(x) is analogous to a cumulative dis- 
tribution function (cdf) from probability theory. These 
mathematical analogies for input-output curves will be 
useful as we continue to analyze the meaning of input- 
output relations and why certain patterns are particularly 
common. 

Note also that k = 1 is the Michaelis-Menten pat- 
tern of chemical kinetics. This relation of the input-output 
curve G(x) to chemical kinetics will be important when 
we connect general aspects of sensitivity to the puzzles of 
chemical kinetics and biochemical input-output patterns. 

The sensitivity is the change in output with respect to 
input. Thus, sensitivity is the derivative of G with respect 
to x, which is 



G(x) 



kx 



k-l 



(1 +x k ) 2 ' 



This expression is analogous to the log-logistic proba- 
bility distribution function (pdf). Here, I obtained the pdf 
in the usual way by differentiating the cdf. Noting that 
the pdf is the sensitivity of the cdf to small changes in 
value (input), we have an analogy between the sensitivity 
of input-output relations and the general relation between 
the pdf and cdf of a probability distribution. 

Maximum sensitivity is the maximum value of G(x), 
which corresponds to the mode of the pdf. For k < 
1, the maximum occurs at x = 0, which means that 
measurement sensitivity of the input-output system is 
greatest when the input is extremely small. Intuitively, 
it seems unlikely that maximum sensitivity could be 
achieved when discriminating tiny input values. For k > 
1, the maximum value of the log-logistic pattern occurs 
when G(x) = 0, which is the point at which the sec- 
ond derivative is zero and the input-output relation is 
purely linear. That maximum occurs at the point given in 
Eq. (12). 

The analogy with probability provides a connection 
between input-output functions, measurement and infor- 
mation. A probability distribution is completely described 
by the information that it expresses [3,40]. That informa- 
tion can be split into two parts. First, certain constraints 
must be met that limit the possible shapes of the distribu- 
tion, such as the mean, the variance, and so on. Second, 
the measurement scale sets the sensitivity of the outputs 
in terms of randomness (entropy) and information (nega- 
tive entropy) in relation to changes in observed values or 
inputs [41,42]. 

Sensitivity, measurement and the shape of input-output 
patterns 

The Hill equation seems almost magical in its ability to fit 
the input-output patterns of diverse biological processes. 
The magic arises from the fact that the Hill equation 
is a simple expression of log-linear-log scaling when the 



Hill coefficient is k > 1. The Hill coefficient expresses 
the locus of linearity. As k declines toward one, the pat- 
tern becomes linear-log, with linearity at low input values 
grading into logarithmic as input increases. As k drops 
below one, the pattern becomes everywhere logarithmic, 
with declining sensitivity as input increases. 

Sensitivity and measurement scale are the deeper 
underlying principles. The Hill equation is properly 
viewed as just a convenient mathematical form that 
expresses a particular pattern of sensitivity, measure- 
ment, and the informational properties of the input- 
output pattern. From this perspective, one may ask 
whether alternative input-output functions provide simi- 
lar or better ways to express the underlying log-linear-log 
scale. 

Frank & Smith [41,42] presented the general relations 
between measurement scales and associated probability 
distribution function (pdf) patterns. Because a pdf is anal- 
ogous to an expression of sensitivity for input-output 
functions, we can use their system as a basis for alterna- 
tives to the Hill equation. Perhaps the most compelling 
general expressions for log-linear-log scales arise from the 
family of beta distributions. For example, the generalized 
beta prime distribution can be written as 



(13) 



With a = k and fS = 1, we obtain a typical form of the 
Hill equation given in Eq. (3). The additional parameters 
a and fi provide more flexibility in expressing different 
logarithmic sensitivities at high versus low inputs. 

The theory of measurement scale and probability in 
Frank & Smith [41,42] also provides a way to analyze more 
complex measurement and sensitivity schemes. For exam- 
ple, a double log scale (logarithm of a logarithm) reduces 
sensitivity below classical single log scaling. Such double 
log scales provide a way to express more extreme dissipa- 
tion of signal information in a cascade at low or high input 
levels. 

These different expressions for sensitivity have two 
advantages. First, they provide a broader set of empirical 
relations to use for fitting data. Those empirical relations 
derive from the underlying principles of measurement 
scale. Second, the different forms express hypotheses 
about how signal processing cascades dissipate informa- 
tion in signals and alter patterns of sensitivity. For exam- 
ple, one may predict that certain signal cascade archi- 
tectures dissipate information more strongly and lead to 
double logarithmic scaling and loss of sensitivity at certain 
input levels. Further theory could help to sort out the pre- 
dicted relations between signal processing architecture, 
the dissipation of information, and the general forms of 
input-output relations. 
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Conclusions 

Nearly all aspects of biology can be reduced to inputs 
and outputs. A chemical reaction is the transformation 
of input concentrations to output concentrations. Devel- 
opmental or regulatory subsystems arise from combina- 
tions of chemical reactions. Any sort of sensory mea- 
surement of environmental inputs follows from chemical 
output responses. The response of a honey bee colony to 
changes in temperature or external danger follows from 
perceptions of external inputs and the consequent output 
responses. Understanding biology mostly has to do with 
description of input-output patterns and understanding 
the processes that generate those patterns. 

I focused on one simple pattern, in which outputs rise 
with increasing inputs. I emphasized basic chemistry for 
two reasons. First, essentially all complex biological pro- 
cesses reduce to cascades of simple chemical reactions. 
Understanding complex systems ultimately comes down 
to understanding the relation between combinations of 
simple reactions and the resulting patterns at the system 
level. Second, the chemical level presents a clear puz- 
zle. The classical theory of chemical kinetics predicts a 
concave Michaelis-Menten input-output relation. By con- 
trast, many simple chemical reactions follow an S-shaped 
Hill equation pattern. The input-output relations of many 
complex systems also tend to follow the Hill equation. 

I analyzed this distinction between Michaelis-Menten 
kinetics and Hill equation patterns in order to illustrate 
the broad problems posed by input-output relations. Sev- 
eral conclusions follow. 

First, many distinct chemical processes lead to the Hill 
equation pattern. The literature mostly considers those 
different processes as a listing of exceptions to the clas- 
sical Michaelis-Menten pattern. Each observed departure 
from Michaelis-Menten is treated as a special case requir- 
ing an explicit mechanistic explanation chosen from the 
list of possibilities. 

Second, I emphasized an alternative perspective. A 
common pattern is widespread because it is consistent 
with the greatest number of distinct underlying mecha- 
nisms. Thus, the Hill equation pattern may be common 
because there are so many different processes that lead to 
that outcome. 

Third, because a particular common pattern associates 
with so many distinctive underlying processes, it is a mis- 
take to treat each observed case of that pattern as demand- 
ing a match to a particular underlying process. Rather, one 
must think about the problem differently. What general 
properties cause the pattern to be common? What is it 
about all of the different processes that lead to the same 
outcome? 

Fourth, I suggested that aggregation provides the proper 
framing. Roughly speaking, aggregation concerns the 
structure by which different components combine to 



produce the overall input-output relations of the system. 
The power of aggregation arises from the fact that great 
regularity of pattern often emerges from underlying dis- 
order. Deep understanding turns on the precise relation 
between underlying disorder and emergent order. 

Fifth, measurement in relation to the dissipation of 
information sets the match between underlying disor- 
der and emergent order. The aggregate combinations of 
input-output processing that form the overall system pat- 
tern tend to lose information in particular ways during 
the multiple transformations of the initial input signal. 
The remaining information carried from input to output 
arises from aspects of precision and measurement in each 
processing step. 

Sixth, previous work on information theory and prob- 
ability shows how aggregation may influence the general 
form of input-output relations. In particular, certain com- 
mon scaling relations tend to set the invariant information 
carried from inputs to outputs. Those scaling relations 
and aspects of measurement precision tell us how to 
evaluate specific mechanisms with respect to their gen- 
eral properties. Further work may allow us to classify 
apparently different processes into a few distinctive sets. 

Seventh, classifying processes by their key properties 
may ultimately lead to a meaningful and predictive the- 
ory. By that theory, we may understand why apparently 
different processes share similar outcomes, and why cer- 
tain overall patterns are so common. We may then predict 
how overall pattern may change in relation to the struc- 
tural basis of aggregation in a system and the general 
properties of the underlying components. More theoreti- 
cal work and associated empirical tests must follow up on 
that conjecture. 

Eighth, I analyzed the example of fundamental chemical 
kinetics in detail. My analysis supports the general points 
listed here. Specific analyses of other input-output rela- 
tions in terms of aggregation, measurement and scale will 
provide the basis for a more general theory. 

Ninth, robustness means insensitivity to perturbation. 
Because system input-output patterns tend to arise by 
the regularities imposed by aggregation, systems natu- 
rally express order arising from underlying disorder in 
components. The order reflects broad structural aspects 
of the system rather than tuning of particular compo- 
nents. Perturbations to individual components will there- 
fore tend to have relatively little effect on overall system 
performance— the essence of robustness. 

Finally, natural selection and biological design may be 
strongly influenced by the regularity of input-output pat- 
terns. That regularity arises inevitably from aggregation 
and the dissipation of information. Those inevitably regu- 
lar patterns set the contours that variation tends to follow. 
Thus, biological design will also tend to follow those con- 
tours. Natural selection may act primarily to modulate 
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system properties within those broad constraints. How 
do changes in extrinsic selective pressures cause natural 
selection to alter overall system architecture in ways that 
modulate input-output patterns? 

Reviewers' comments 

Reviewer's report 1 

Eugene Koonin, NCBI, NLM, NIH, United States of America 

In my view, this is an important, deep analysis that con- 
tinues the series of insightful studies by the author in 
which various aspects of the manifestation of the Maxi- 
mum Entropy principle are investigated. In this particular 
case, biological systems are looked at from the standpoint 
of input-output relations, and it is shown how informa- 
tion dissipation caused by aggregation of signals from 
numerous components of biological systems leads to com- 
mon patterns such as the Hill equation. The complex 
relationship between patterns and processes is empha- 
sized whereby the universal patterns, such as that given by 
the Hill equation, are so common because highly diverse 
underlying processes can converge to produce these pat- 
terns. It is further emphasized that natural selection is 
likely to act primarily as a modulator on the regular 
patterns yielded by aggregation and dissipation of infor- 
mation. In short, an important paper that, together with 
the previous publications of the author, should come as an 
important revelation to many biologists. One only hopes 
at least some of them read. 

Reviewer's report 2 

Georg Luebeck, Fred Hutchinson Cancer Research Center, 
United States of America 

The review of molecular input-output relations in biolog- 
ical systems by Dr. Frank, although broad and somewhat 
speculative, wrestles with a fundamental problem in Sys- 
tems biology: with what Nobel laureate Sydney Brenner 
referred to as ". . . low input, high throughput, no out- 
put science". The perspective Frank offers in this review 
is refreshing and instructive. Rather than taking on the 
"inverse-problem" Brenner alludes to by its horns, Frank 
dissects the characteristics of commonly seen biological 
input-output relations in view of measurement error, sig- 
nal processing, information loss, and system aggregation. 
His analysis is motivated by an apparent contradiction 
between classical Michaelis-Menten kinetics and the S- 
shaped Hill (equation) response often seen in biological 
processes. The resolution of this apparent paradox along 
various lines of thought and argument, accompanied by 
easy-to-understand examples, are enlightening and in 
themselves worth reading. 

The review offers many interesting tidbits related to 
measurement, psychophysics, and information process- 
ing. However, the development of the main points feels 



somewhat disorganized and random in their order. For 
example, for the uninitiated reader, it is not immediately 
clear until later that the sensitivity imparted on an out- 
put signal by the input is logarithmic for the Hill equation 
with k > 1. For an initiated reader, however, that may be 
entirely trivial. Still, it seems better to demonstrate such a 
crucial point upfront. 

As it stands, the review could benefit from some short- 
ening and additional focusing, something I trust the 
author will attempt. Although the main points raised by 
Dr. Frank are clearly presented in general, he often seems 
to be one or two steps ahead of the reader who struggles 
to make sense of formulations that do not become clear 
until later. For example, early on, he refers to "the hap- 
hazard combination of different scalings in the distinct 
reactions . . ." or the "transmission and loss of informa- 
tion in relation to scale..." without first defining what 
he means by 'scale' (or scaling) and elaborating on how 
measurement models — that conform to the input-output 
paradigm — actually reveal scaling. The example he offers 
later on provides some intuition, but misses the point as 
it fails to address the role of noise (fluctuations in input 
signals) in setting the scale (at low inputs), by reducing 
an increasing number of false positives as genuine signals 
may get drowned out by noise. As the author points out, 
there is plenty of room for advancing development and 
understanding. It is exactly at the intersection of optimal 
information control and processing that guided aggrega- 
tion of molecular processes leads to emerging order and 
the (controlled) biological complexity we call life. Perhaps, 
if we understand the invariant properties of the molec- 
ular processes that participate in life, and the common 
biophysical principles that guide the evolution of biologi- 
cal systems, we may be able to get on top of the "inverse 
problem" that Brenner so bleakly refers to. The alternative, 
solving the "forward problem" by brute-force computation 
is tedious and not very satisfying from an intellectual point 
of view. 
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Sergei Maslov, Brookhaven National Laboratory, United 
States of America 
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